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ABSTRACT 

A new method for spectral-spatial classification of hyper- 
spectral images is proposed. The method is based on the 
integration of probabilistic classification within the hierarchi- 
cal best merge region growing algorithm. For this purpose, 
preliminary probabilistic support vector machines classifica- 
tion is performed. Then, hierarchical step-wise optimization 
algorithm is applied, by iteratively merging regions with 
the smallest Dissimilarity Criterion (DC). The main novelty 
of this method consists in defining a DC between regions 
as a function of region statistical and geometrical features 
along with classification probabilities. Experimental results 
are presented on a 200-band AVIRIS image of the North- 
western Indiana’s vegetation area and compared with those 
obtained by recently proposed spectral-spatial classification 
techniques. The proposed method improves classification 
accuracies when compared to other classification approaches. 

Index Terms — Hyperspectral images, classification, seg- 
mentation, region growing, support vector machines. 

1. INTRODUCTION 

Hyperspectral imaging [1], which acquires hundreds of spec- 
tral channels for each pixel, opens new perspectives in clas- 
sification of remote sensing images. While pixelwise clas- 
sification methods process each pixel independently without 
considering the correlations between spatially adjacent pix- 
els [2, 3], recent studies have shown the advantage of in- 
cluding information from a spatial neighborhood for accu- 
rate classification, i.e., performing spectral-spatial classifica- 
tion [4, 5]. 

In previous works, we have proposed to perform a seg- 
mentation of the input image and then use the identified 
segments as pixels’ neighborhoods [6]. However, automatic 
segmentation of hyperspectral images is a challenging task, 
since its performance strongly depends on the chosen mea- 
sure of region homogeneity. Hierarchical multilevel defini- 
tion of regions (achieved by performing iterative best merge 
region growing) mitigates this dependence [7], but this ap- 
proach is not completely automated (the user is required to 
set the parameter defining the multilevel segmentation) and 


computationally expensive. An alternative spectral-spatial 
classification method consists in performing probabilistic 
classification for selecting the most reliably classified pixels 
as markers for segmentation [5], In this case, the choice of 
markers strongly depends on the performance of the selected 
classifier. 

For both approaches, segmentation and classification 
techniques were applied one after another, and their outputs 
were combined in some way. In this work, we aim to inter- 
lace these procedures and develop a new methodology, where 
segmentation and classification are performed concurrently 
for obtaining an accurate thematic map. For this purpose, 
a new Hierarchical Segmentation with integrated Classifi- 
cation (HSwC) method for hyperspectral data is proposed. 
First, preliminary probabilistic classification of each pixel 
is performed. Then, best merge region growing is applied, 
where at each iteration two “closest regions’’ are merged, and 
classification probabilities for a new region are recomputed. 
The main idea behind this new method consists in defining 
a Dissimilarity Criterion (DC) between regions as a function 
of region statistical features, classification probabilities and 
geometrical features. The algorithm is converged when all 
the pixels get a definite classification label. The output of 
the proposed HSwC method is a thematic map, where every 
region is assigned to one of the classes of interest. 

The paper is organized as follows. In the next section, a 
new HSwC approach is presented. Experimental results are 
discussed in Section 3. Finally, conclusions are drawn in Sec- 
tion 4. 


2. PROPOSED CLASSIFICATION APPROACH 

The flowchart of the proposed HSwC classification method is 
shown in Fig. 1. An input /j-haud hyperspectral image can 
be considered as a set of n pixel vectors X = {xj £ j = 
1, 2, ..., n}. The objective is to compute a classification map 
L = {Lj,j = 1, 2, ..., n}, where each pixel Xj is assigned to 
one of K information classes (i.e., has a class label Lj). The 
following procedure is proposed for this purpose: 





Fig. 2. Flowchart of computing a dissimilarity criterion be- 
tween two regions /?, and Rj . 


Fig. 1. Flowchart of the proposed HSwC classification 
method. “DC” means Dissimilarity Criterion. 


2.1. Preliminary probabilistic classification 

The first step consists in performing a preliminary probabilis- 
tic classification of the hyperspectral image. We propose to 
use a Support Vector Machines (SVM) classifier for this pur- 
pose which has shown a good performance for classifying hy- 
perspectral data [3]. The standard SVM classifier does not 
provide probability estimates for the individual clases. In or- 
der to estimate these estimates, pairwise coupling of binary 
probabilistic estimates is applied [5]. This step results in a 
classification map L' = {i',j = 1, 2, ..., n}, where each 
pixel has a unique class label, and n vectors of class prob- 
abilities for each pixel {P(Lj = k\xj),k = 1 = 

1)2, 

2.2. Hierarchical segmentation with integrated classifica- 
tion 

At the next step. Hierarchical Step-Wise Optimization seg- 
mentation (HSWO) [8] with integrated classification is per- 
formed as follows: 

1) Initialize the algorithm by assigning a new region la- 
bel for each pixel. Each one-pixel region Ri gets a prelim- 
inary class label L(i?,) and a vector of class probabilities 

{P k (Ri) = P(L(Ri) = k\Ri),k = 1, ..., K}. 


2) Calculate the DC between all pairs of spatially adjacent 
regions. We propose to compute a DC between two regions 
R, and Rj as a function of region statistical (mean vector) and 
geometrical (number of pixels) features along with classifica- 
tion probabilities (see Fig. 2): 


• Compute the Spectral Angle Mapper (SAM) measure 
between the region mean vectors Uj = (un, ..., Mi_e) T 
and ir, = (uji , ..., Ujb) t as 


SAM (u t , u,) = arccos 


Eb=l u ib u jb 


Ef=i^ 1 / 2 Ef=i ^] 1/2 


(i) 


• If the regions have equal class labels = L(Rj) = 

k', 

DC(Ri, Rj) = (2 — max(Pfc'(f?i)) -Pfc' (Rj)))SAM(\ii,Uj). 

( 2 ) 


• Otherwise (i.e., if L(Ri) ^ L(Rj)), if a number of pix- 
els in each region is larger than M, DC (II, . Rj) = oo 
(in practice, the upper maximum value of float). This 
condition is included for favoring merging small re- 
gions. We propose to set M = 20. 


• Otherwise, 

DC{Ri,Rj) = 

(2 - min(P L(flj) (A J ),PL(fl I )(i? J )))5AM(u ! ,u J ). (3) 

3) Find the smallest DC value DC m i n . 









4) Merge all pairs of spatially adjacent regions with 
DC = DC mi n . For each new region R new created by 
merging two regions //, and R :) , recompute: 

• A vector of new class probabilities as 

D , D ^ _ Pk{Ri)card{Ri ) + PkiR^cardiRj) 

DkyRnew) — ,, r> \ ) (4) 

card(R new ) 

k = 1, ...,K, where ca r di ll ) is a number of pixels in the 
region R, card(R new ) = card(Ri) + card(Rj). 

• Class label as 

R(Rnew') — arg max j F). ( Rrieu : ) } ■ (5) 

k 

All the pixels in the new region R new get a definite class 
label (i.e., if R, and/or //, were composed of one pixel with a 
preliminary classification label, the corresponding pixels get 
a definite label of the class). 

5) Stop if all the pixels (n pixels) get a definite classifi- 
cation label. If the considered image may have one -pixel re- 
gions of interest, the algorithm is converged when P ■ n pixels 
(0 < P < 1) are definitely classified. If the algorithm is not 
converged, recalculate the DC values for the new regions and 
all regions spatially adjacent to them, and return to step 3. 

3. EXPERIMENTAL RESULTS AND DISCUSSION 

We applied the proposed method to the Indian Pines image 
recorded by the AVIRIS (Airborne Visible/Infrared Imaging 
Spectrometer) sensor over the vegetation area in Northwest- 
ern Indiana. The image is of 145 by 145 pixels, with a spatial 
resolution of 20 m/pixel. Twenty water absorption bands have 
been removed, and a 200 -band image was used for experi- 
ments. Sixteen classes of interest are considered, which are 
detailed in Table 1, with a number of training and test samples 
for each class. Training samples have been randomly selected 
from the reference data. More information about the image 
and reference data can be found in [ 6 ], 

A probabilistic one-versus-one SVM classification with 
the Gaussian Radial Basis Function (RBF) kernel was per- 
formed. The optimal parameters were chosen by fivefold 
cross validation: penalty during the SVM optimization 

C = 128, spread of the RBF kernel 7 = 2~ 6 . Classifi- 
cation map is shown in Fig. 3(a). Then, the proposed HSwC 
method was applied (the algorithm was implemented using 
the Hierarchical SEGmentation (HSeg) software [9]). The 
algorithm has been converged when: a) 99% of pixels were 
classified (P = 0.99); b) all pixels were classified (P = 1). 

Table 1 summarizes global (overall, average accuracies 
and kappa coefficient [ 6 ]) and class-specific accuracies of 
the SVM classification and the proposed HSwC technique 
(with P = 0.99 and P = 1). In order to compare the ef- 
ficiency of the proposed method with previously proposed 
techniques, we have included results of the ECHO classifica- 
tion [ 10 ], a classification using the construction of a minimum 



(a) (b) 


Fig. 3. Indian Pines image, (a) SVM classification map. (b) 
HSwC classification map ( P = 1). 

spanning forest from the probabilistic SVM-derived markers 
(, SVMMSF ) [5] and a classification by majority voting within 
neighborhoods defined by HSeg segmentation (HSeg+MV, 
with the parameter S wg ht = 0 . 0 , which is equivalent to 
HSWO, and the SAM DC) [ 6 ], 

As can be seen from Table 1, accuracies are significantly 
improved when including spatial information into the classi- 
fication. The proposed HSwC method, with P = 1, yields the 
best average accuracy (which is improved by 8.2 percentage 
points when compared to the SVM classification) and most of 
the class-specific accuracies. Fig. 3(b) shows the correspond- 
ing classification map, which is much less noisy than a pixel- 
wise classification map. The HSwC algorithm with P = 0.99 
yields lower classification accuracies. Therefore, unless there 
are one-pixel regions of interest in the image under consider- 
ation, we recommend to run the algorithm until all the pixels 
get a definite classification label (i.e., P = 1). 

The best overall accuracy is obtained using the HSeg+MV 
technique. However, in this method a segmentation map at an 
appropriate level of segmentation detail was chosen by quan- 
titatively evaluating classification accuracies at several hierar- 
chical levels versus the test data and retaining the best results. 
The test data are not available in most cases for classifying a 
new data set. Furthermore, the HSwC approach proposed in 
this paper requires less parameters to tune when compared to 
the ECHO and SVMMSF techniques. The only parameters to 
be tuned are C and 7 for the SVM classification, a parameter 
M for favoring merging small regions (which is intuitive to 
tune), and a parameter P if required. 

4. CONCLUSIONS 

Hyperspectral imagery is a well-suited technology for accu- 
rate image classification, which is an important task in many 
application sectors (such as precision agriculture, monitoring 
and management of the environment, etc.). A new HSwC 
method for spectral-spatial classification of hyperspectral data 
was presented in this paper. In this method, a preliminary 
probabilistic classification is first applied, using an SVM clas- 




Table 1. Information Classes, Number of Labeled Samples (No. of Samp.) and Classification Accuracies in Percentage for the 
Indian Pines Image. 



No. of Samp. | 

SVM 

ECHO 

SVM 

HSeg 

HSwC 


Train 

Test 

MSF 

+MV 

P = 0.99 

P= 1 

Overall Accuracy 

- 

- 

78.17 

82.64 

88.41 

90.86 

87.95 

89.24 

Average Accuracy 

- 

- 

85.97 

83.75 

91.57 

93.96 

92.91 

94.18 

Kappa Coefficient k 

- 

- 

75.33 

80.38 

86.71 

89.56 

86.31 

87.76 

Corn-no till 

50 

1384 

78.18 

83.45 

90.97 

90.46 

92.20 

93.06 

Corn-min till 

50 

784 

69.64 

75.13 

69.52 

83.04 

81.63 

82.53 

Corn 

50 

184 

91.85 

92.39 

95.65 

95.65 

95.65 

97.28 

Soybeans-no till 

50 

918 

82.03 

90.10 

98.04 

92.06 

94.44 

95.10 

Soybeans-min till 

50 

2418 

58.95 

64.14 

81.97 

84.04 

72.54 

74.36 

Soybeans-clean till 

50 

564 

87.94 

89.89 

85.99 

95.39 

93.62 

96.10 

Alfalfa 

15 

39 

74.36 

48.72 

94.87 

92.31 

92.31 

97.44 

Grass/pasture 

50 

447 

92.17 

94.18 

94.63 

94.41 

91.28 

93.96 

Grass/trees 

50 

697 

91.68 

96.27 

92.40 

97.56 

97.85 

97.85 

Grass/pasture-mowed 

15 

11 

100 

36.36 

100 

100 

100 

100 

Hay-windrowed 

50 

439 

97.72 

97.72 

99.77 

99.54 

98.86 

98.86 

Oats 

15 

5 

100 

100 

100 

100 

100 

100 

Wheat 

50 

162 

98.77 

98.15 

99.38 

98.15 

99.38 

99.38 

Woods 

50 

1244 

93.01 

94.21 

97.59 

98.63 

98.39 

99.52 

Bldg-Grass-Tree-Drives 

50 

330 

61.52 

81.52 

68.79 

82.12 

78.48 

81.52 

Stone-steel towers 

50 

45 

97.78 

97.78 

95.56 

100 

100 

100 


sifier. Then, iterative best merge region growing with in- 
tegrated classification is performed. The main contribution 
consists in defining a DC between adjacent regions as a func- 
tion of region statistical and geometrical features and classi- 
fication probabilities. Experimental results did show that the 
proposed approach improves classification accuracies, when 
compared to previously proposed classification techniques, 
and provides classification maps with more homogeneous re- 
gions. Similar results are obtained for other datasets acquired 
by the ROSIS (Reflective Optics System Imaging Spectrom- 
eter) sensor. 

In conclusion, the proposed approach succeeded in taking 
advantage of the spatial and spectral information simultane- 
ously for accurate hyperspectral data classification. 
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